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CONTROLLING OPERATION OF A DEVICE 
USING A RE-CONFIGURABLE WATERMARK 
DETECTOR 

RELATED APPLICATION DATA 

[0001] This application is a continuation of co-pending 
application Ser. No. 09/574,668, filed May 18, 2000, which 
claims priority to provisional patent application No. 60/134, 
782, filed May 19, 1999, incorporated herein by reference. 

[0002] The subject matter of the present application is 
related to that disclosed in application Ser. No. 09/234,780, 
filed Jan. 20, 1999; Sen No. 09/314,648, filed May 19, 1999; 
Ser. No. 09/337,590, filed Jun. 21, 1999; Ser. No. 09/433, 
104, filed Nov. 3, 1999; Ser. No. 09/441,819, filed Nov. 17, 
1999; Ser. No. 09/441,821, filed Nov. 17, 1999; Ser. No. 
09/442,441, filed Nov. 17, 1999; Ser. No. 09/464,307, filed 
Dec. 15, 1999; Ser. No. 09/473,075, filed Dec. 28, 1999; Ser. 
No. 09/476,686, filed Dec. 30, 1999; Ser. No. 09/482,752, 
filed Jan. 13, 2000; Ser. No. 09/484,742, filed Jan. 18, 2000; 
Ser. No. 09/498,223, filed Feb. 3, 2000; No. 60/180,364, 
filed Feb. 4, 2000; Ser. No. 09/520,406, filed Mar. 8, 2000; 
Ser. No. 09/563,664, filed May 2, 2000, and Ser. No. 
09/562,517, filed May 1, 2000. 

FIELD OF THE INVENTION 

[0003] The present invention relates to applications of 
digital watermarking in conjunction with audio, video, 
imagery, and other media content. 

BACKGROUND AND SUMMARY 

[0004] Watermarking (or "digital watermarking") is a 
quickly growing field of endeavor, with several different 
approaches. The present assignee's work is reflected in U.S. 
Pat. No. 5,862,260; in copending application Ser. Nos. 
09/503,881 and 09/452,023; and in published specifications 
WO 9953428 and WO0007356 (corresponding to U.S. Ser. 
Nos. 09/074,034 and 09/127,502). A great many other 
approaches are familiar to those skilled in the art. The artisan 
is presumed to be familiar with the full range of watermark- 
ing literature. 

[0005] In the present disclosure it should be understood 
that references to watermarking encompass not only the 
assignee's watermarking technology, but can likewise be 
practiced with any other watermarking technology. 

[0006] The invention relates to methods, devices and 
systems for controlling operation of a device using a re- 
configurable watermark detector. In many applications, it is 
useful to be able to change the operation of a watermark 
detector. Such changes may include changing how the 
watermark detector decodes or interprets a watermark 
embedded in a signal of a given media type, such as audio, 
video or still images. 

[0007] The ability to reconfigure a watermark detector is 
advantageous in many applications. When watermark detec- 
tors are deployed widely in consumer electronic devices and 
software, the capability to reconfigure the watermark detec- 
tor reduces or prevents the devices and software from 
becoming obsolete when changes to the detector are needed. 
In content management schemes, where the watermark is 
used as a security feature, the security mechanism may be 
hacked. In other applications, the watermark technology 



provider may want to incorporate new features into the 
watermark detector, expand the message payload of the 
watermark, or change how the watermark payload is inter- 
preted. In these circumstances, it is advantageous to be able 
to reconfigure a watermark detector, and particularly, to 
reconfigure the detector remotely. 

[0008] One aspect of the invention is a method for con- 
trolling the operation of a device using a watermark. The 
method decodes a watermark embedded in a signal of a 
given media type, such as an image or audio signal (e.g., still 
images, motion pictures, audio, video, etc.). The watermark 
includes a command signal used to trigger a change in 
operation of the watermark detector. Based on the command 
signal, the method changes the operation of the watermark 
detector. This change may include changing how the water- 
mark detector decodes or interprets a watermark in a signal 
of the media type. After changing operation of the water- 
mark detector, the method decodes a usage control restric- 
tion from a watermark embedded in the signal. 

[0009] The scope of this method encompasses a variety of 
implementations. The command signal may be represented 
as one or more bits of a watermark payload carried by the 
watermark. Changing the operation of the detector may 
include re-programming it or altering how it interprets 
watermark data embedded in a media object. For example, 
the method may transfer firmware instructions to the detec- 
tor to replace instructions stored earlier. As another example, 
the command may change the operation of the detector 
according to a preprogrammed rule. The rule may define a 
change in watermark key, for instance. Yet another example 
is changing the operation of the detector by changing how a 
device responds to the watermark signal extracted from a 
media object. The behavior of a hardware or software media 
player, for instance, may be updated to respond differently to 
the watermark signal, and specifically, to the message car- 
ried in a watermark payload. 

[0010] The method may use watermark payload data to 
change the operation of the detector. For example, the 
payload may specify instructions or watermark key data. In 
response to this payload data, the detector may install and 
execute new instructions or use the new watermark key to 
decode watermarks in media objects. 

[0011] Another aspect of the invention is an alternative 
method for controlling the operation of a device using a 
watermark detector. This method receives a media object 
and a command associated with the media object signaling 
that the watermark detector requires an upgrade to decode a 
watermark from the media object. In response to the com- 
mand, the method updates the watermark detector to create 
an updated watermark detector. It decodes the watermark 
from the media object with the updated watermark detector, 
including decoding a usage control restriction from the 
watermark. The command may be encoded in a watermark 
in the media object, or may be conveyed in a channel 
different than the watermark channel yet transmitted along 
with the media object (e.g., out-of-band channels like a file 
header or footer, sub-titling data channel, SCA channel, 
etc.). Thcscope of this method encompasses a variety of 
implementations. In addition, aspects of this method may be 
used in combination with aspects of the method summarized 
earlier. 

[0012] Yet another aspect of the invention is a re-program- 
mable watermark detector. The detector comprises a water- 
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mark decoder for detecting a command to upgrade the 
detector. It also includes instructions that are replaceable in 
response to detecting the command to upgrade the detector. 
The detector instructions are operable to decode a usage 
control restriction from a watermark and control playback or 
recording of a media object carrying the watermark. The 
instructions may be conveyed to the watermark detector 
along with a media object, either in a watermark payload, or 
a channel other than the watermark channel. 

[0013] Another aspect of the invention is a method of an 
encoding an upgrade trigger in a watermark. The method 
receives a media object of a given media type, and encodes 
a watermark into the media object. The watermark includes 
a command signal used to trigger a change in operation of 
a watermark detector. When received in a detector, this 
change operates to alter how the watermark detector decodes 
or interprets a watermark in a signal of the media type. In 
addition, the method encodes a usage control restriction into 
the watermark, where the change in the operation of the 
detector enables the watermark detector to read the usage 
control restriction. 

[0014] Further features and aspects of the invention will 
become apparent with reference to the following detailed 
description and accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0015] FIG. 1 is a diagram showing the participants, and 
channels, involved in the distribution of music. 

[0016] FIG. 2 shows a conceptual model of how music 
artists, record labels, and E-Music distributors can all inter- 
act with a Media Asset Management System, of which 
several are detailed in the following specification. 

DETAILED DESCRIPTION 

[0017] For expository convenience, much of the following 
discussion focuses on music, but the same principles and 
techniques are largely or wholly applicable to other source 
data, whether non-music audio, video, still imagery, printed 
materials, etc. 

[0018] Music Asset Management 

[0019] Referring to the figures, the music distribution 
process begins with a creative artist 10. The artist's music 
has traditionally been distributed by a record label 12. 
(While the following discussion refers to distribution 
through such a label, it should be understood that such 
distribution can just as well be effected directed under the 
artist's control, without a record label intermediary.) 

[0020] In traditional distribution 14, the record label pro- 
duces tangible media, such as records, tapes, videos (e.g. 
music videos), and CDs 16. These media are physically 
distributed to end-consumers 18. Additionally, the label 12 
distributes the music media to outlets 20, such as radio and 
TV stations, cable and satellite systems, etc., which broad- 
cast (or narrowcast) the artist's work to an audience. Dis- 
tribution through such media outlets may be monitored by 
playout tracking services. Playout tracking data, collected by 
firms including Arbitroo, Nielsen, ASCAP, BMI, etc., can be 
used to compute royalty payments, to verify broadcast (e.g. 
for advertising), etc. 



[0021] Increasingly, the distribution of the music to the 
media outlets is performed electronically. Such distribution 
first took the form of analog audio over high quality land- 
lines or satellite channels. Digital audio quickly supplanted 
analog audio in such distribution channels due to higher 
fidelity. 

[0022] More recently, distribution of the music from the 
record labels to the media outlets has occurred over secure 
links, now including the internet. Such security was first 
provided simply by scrambling the audio signal or data. 
More sophisticated "containers'-based systems are now com- 
ing into vogue, in which the audio is "packaged" (often in 
encrypted form) with ancillary data. 

[0023] Electronic distribution of music to the consumer is 
also gaining popularity, presently in the MP3 format prima- 
rily. The music providers may deal directly with the public, 
but more commonly effect such consumer distribution 
through a newly emerging tier of digital media outlets, such 
as internet sites that specialize in music. From such sites, 
consumers can download digital audio files into personal 
digital audio players. (The Diamond Rio, and the Audible 
MobilePlayer devices are some of the first of what will 
doubtless be a large number of entrants into this personal 
internet audio appliance market.) Or the downloaded data 
can be stored by the consumer-recipient onto any other 
writeable media (e.g. hard disk, CD, DVD, tape, videotape, 
etc.). Typically a personal computer is used for such down- 
loading, but this intermediary may be dispensed with by 
coupling next generation of personal audio appliances to an 
internet-like link. 

[0024] The data downloaded by the consumer can be 
stored either in the native digital format, translated into 
another digital format (which translation may include 
decryption), converted into analog and recorded in analog 
form, etc. 

[0025] Unauthorized copying or use of the music can 
occur anywhere in the foregoing channels. However, one of 
the greatest risks occurs once the music has been delivered 
to the consumer (whether by tangible media, by traditional 
broadcast media outlets, by emerging digital distribution, or 
otherwise). 

[0026] The general idea of embedding auxiliary data into 
music (i.e. watermarking) has been widely proposed, but so 
far has been of limited applicability. 

[0027] For example, GoodNoise is planning to embed a 
digital signature — termed a multimedia identifier, or MMI — 
in its MP3 music. MMI will register the song and its author 
with a licensing number. In addition to providing informa- 
tion about the songwriter and distributor, this digital encod- 
ing may also include lyrics, liner notes, and other informa- 
tion. But all of the proposed uses serve only to convey 
information from the distributor to the consumer; use for 
"tracking" is actively disclaimed. (Wired News, "Good- 
Noise Tags MP3 Files," Feb. 3, 1999.) 

[0028] TJie Genuine Music Coalition — a partnership of 
various companies in the music distribution business — 
likewise has announced plans to employ watermarking of 
MP3 music. The watermarking technology, to be provided 
by Liquid Audio, will convey data specifying the artist or 
producer contact, copyright data, and a number to track 
ownership. The Coalition hopes that the provision of this 



07/18/2003, EAST Version: 1.03.0002 



US 2002/0012443 Al 



3 



Jan. 31, 2002 



embedded information will help thwart piracy. Industry 
observers believe Liquid Audio will next introduce playback 
technology only plays audio in which its watermark is 
detected. (Wired News, "Liquefying MP3," Jan. 23, 1999.) 

[0029] A similar initiative has been announced by the 
Recording Industry Association of America (RIAA). Termed 
the Secure Digital Music Initiative (SDMI), the program 
seeks to define a voluntary specification that will assure 
proper compensation to those who produce and distribute 
music. One element of the system will likely be a water- 
marking component. (Dow Jones Newswire, "Spurred By 
Maverick Technology, Music Industry Eyes Web," Dec. 31, 
1998.) 

[0030] Yet another initiative has been announced by 
Solana and ASCAP. Other companies promoting watermark- 
ing for music include Axis Technology, MCY.com, and 
AudioSoft. 

[0031] The watermark pay load can represent various types 
of data. An exemplary payload includes data relating to the 
artist, distribution entity, title, and copyright date/proprietor. 
Additionally, the payload can include a digital object iden- 
tifier — an ISBN-like number issued by a central organiza- 
tion (e.g. a rights management organization) to uniquely 
identify the work. 

[0032] Such payload data can be encoded literally (e.g. the 
title by a series of ASCII characters, etc.). In other embodi- 
ments, codes or abbreviations can be employed — with each 
code having a known meaning. In still other embodiments, 
the data can be meaningless by itself, but may serve as a key 
(e.g., a Unique Identifier, or UID) into a remote data 
database or repository. An example of such a remote data 
repository is a web site at a Master Global Address (MGA) 
associated with content, as detailed below. 

[0033] An exemplary data payload may, for example, have 
the following format: 



lAIBICIDIEIPIGlHhl I I 



[0034] Where A is a six-byte (8-bits to a byte) ASCII string 
serving as a digital object identifier (which may serve as a 
link to a Master Global Address through a default name 
server, as discussed below), B is a two-byte ASCII field 
serving as a key into an "artist" field of the remote database, 
C is a three-byte ASCII field serving as a key into a "title" 
field of the remote database; D is a 14-bit field serving as a 
key into a "label" field of the remote database, E is an 8-bit 
integer representing the work's year of first publication 
(with 0 representing the year 2000); F is a 10-bit field 
serving as a key into a "price" field of the remote database, 
G is a two-byte usage control string (detailed below), H is 
a streaming data channel, and I is a string of bits serving as 
a cyclic redundancy checksum for the foregoing. (More 
sophisticated error correcting checksums can, of course, be 
employed.) This payload format totals 136 bits, exclusive of 
the CRC coding and the streaming data channel. 

[0035] This payload is encoded repeatedly, or redundantly 
through the music, so that the full payload can be decoded 
from partial excerpts of the music. 



[0036] The encoding is also desirably perceptually adap- 
tive, so that higher energy encoding is employed where the 
listener is less likely to perceive the additional "noise" 
introduced by the encoding, and vice versa. Various tech- 
niques for perceptually adaptive encoding are known. For 
example, some tie the amplitude of the encoded signal to the 
instantaneous amplitude of the music. Others exploit psy- 
choacoustic "masking" of one signal by a spectrally- or 
temporally-adjoining signal of higher energy. Still other 
approaches fill gaps in the music's spectrum with watermark 
energy. 

[0037] In other embodiments, perceptually adaptive 
encoding is not used. In some such embodiments, no tailor- 
ing of the temporal or spectral characteristics of the water- 
mark signal is employed. In others, the watermark signal is 
spectrally filtered to emphasize low frequency audio com- 
ponents (e.g. less than 500 hz), high frequency audio com- 
ponents (e.g. higher than 2500 hz), or mid -frequency audio 
components (500-2500 hz). 

[0038] The streaming data field channel (H) is a medium 
by which data can be conveyed from a distribution site (or 
other site) to the end user. Such data may be entirely 
unrelated to the underlying work. For example, it may serve 
a utilitarian purpose, such as conveying data to a memory in 
the consumer device to replace previously-stored data that is 
out-of-date. It may be a commercial channel on which 
bandwidth is sold for access to the consumer or the con- 
sumer's device. Essentially any purpose can be served by 
this streaming data field. Unlike most of the other fields, the 
streaming data field may not endlessly repeat the same data, 
but can convey data that changes with time. 

[0039] Desirably, the encoding is performed in a manner 
permitting recovery of the watermark data even if the audio 
is corrupted, e.g. by format conversion, re-sampling, tape 
wow and flutter, compression, coding, or various forms of 
audio processing (e.g. filtering, pre-emphasis, re-scaling, 
etc.). One way to provide for such robustness is to encode a 
signal of known character that can be recognized through all 
such corruption. By identifying such known signal, the 
watermark signal can then be decoded. (The known signal 
can take various forms, e.g. a synchronization signal, a 
marker signal, calibration signal, a universal code signal as 
described in applicant's patents, etc.) 

[0040] In some embodiments, a watermark "dial-tone" 
signal is provided. This dial-tone signal is a low amplitude, 
relatively wideband, repetitive signal that commonly con- 
veys only limited information (e.g. a single bit of informa- 
tion). Its presence in an audio signal can serve as a "do not 
record," or similar instruction signal. Alternatively, or in 
addition, the dial-tone signal can serve as an aid in "locking" 
to a plural -bit digital watermark signal that is also encoded 
in the audio. For example, the cyclical repetition of the 
signal can serve to identify the start of the plural-bit digital 
watermark signal. Or the spectrum or repetition rate of the 
signal can identify any temporal corruption of the audio. An 
exemplary such signal is detailed as a "simple universal 
code" in U*.S. Pat. No. 5,636,292. 

[0041] A track of music can be pre-authorized for speci- 
fied types of use. For example, the usage control string of the 
watermark payload may include a six-bit field detailing the 
classes of devices for which the audio is authorized. Each bit 
would correspond to a different class of device. Class 1 
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devices may be personal playback devices with only analog- 
audio output. Class 2 devices may be personal entertainment 
devices capable of outputting music in digital (e.g. MP3, 
redbook, VWAV) format, as well as analog audio. Class 3 
devices may be personal computer systems (i.e. with essen- 
tially unlimited ability for processing and outputting digital 
audio). Etc., etc. A device to which such MP3 audio is 
provided would check the usage control string data to 
determine whether it is authorized to utilize the audio. A 
personal playback device with analog-only output, for 
example, would examine the first bit of the usage control 
string. If it was "1," the device would be authorized to use 
(i.e. playback) the MP3 data; if it was a "0," the device 
would refuse to play the music. 

[0042] In addition to pre-authorization for certain classes 
of devices, the usage control string can also include bits 
indicating the number of permitted playbacks. This data can 
be encoded in bits seven through nine, representing eight 
possibilities: 

[0043] 0 — no playback permitted 

[0044] 1 — single playback permitted 

[0045] 2 — two playbacks permitted 

[0046] 3 — three playbacks permitted 

[0047] 4 — four playbacks permitted 

[0048] 5 — five playbacks permitted 

[0049] 6—10 playbacks permitted 

[0050] 7 — unlimited playbacks permitted 

[0051] 8 — refer to associated data (within the water- 
mark, or stored at a remote site) which specifies 
number of permitted playbacks. 

[0052] The playback device may include a non-volatile 
store in which the number of permitted playbacks is stored 
for each track of music. The device would decrement this 
number at the beginning of each playback. 

[0053] The usage control string can also include a two-bit 
field (bits ten and eleven) indicating recording permissions. 
A value of 0 means that data corresponding to the MP3 audio 
(regardless of digital format) should never be made available 
to another digital device. A value of 1 means that the data 
corresponding to the MP3 data may be made available once 
to another digital device. A value of 2 means that the data 
may be made available an unlimited number of times to 
other digital devices. (Value 3 is reserved.) 

[0054] Another data field that can be included in an audio 
watermark is a rating that indicates age -appropriateness. 
Music with violence or sexual themes might be given a 
rating akin to the MPAA "PG-13" or "R" rating. Audio 
appliances may be programmed to recognize the rating of 
incoming music, and to interrupt playback if the rating 
exceeds a certain threshold setting. Various known tech- 
niques can be employed to assure that such settings cannot 
readily be changed, e.g., by juvenile listeners. 

[0055] Another data field that can be included in an audio 
watermark is a date field. This field can indicate either the 
date the music was watermarked, or a date in the future on 
which certain rights associated with the music should 
change. Some consumers, for example may not wish to 



purchase perpetual playback rights to certain musical selec- 
tions. The right to play a selection for 6 months may suffice 
for many consumers, especially if the price is discounted in 
view of the limited term. Such an arrangement would not be 
wholly disadvantageous to music distributors, since some 
consumers may end up purchasing music twice if their initial 
assessment of a musical selection's appeal was too short- 
sighted. (Naturally, the playback equipment would require a 
source of real-time clock data against which the date field in 
the watermark can be checked to ensure that the playback 
rights have not yet expired.) 

[0056] Another of the data fields that can be included in an 
audio watermark specifies technical playback parameters. 
For example, the parameter can cause the playback appli- 
ance to apply a spectral equalization that favors bass fre- 
quencies, or treble frequencies, or mid-range frequencies, 
etc. Other pre-configured equalization arrangements can 
similarly be invoked responsive to watermark data. Like- 
wise, the parameter can invoke special-effects provided by 
the playback appliance, e.g., echo effects, reverb, etc. 
(Again, such parameters are usually represented in an abbre- 
viated, coded form, and are interpreted in accordance with 
instructions stored in a memory (either in the playback 
appliance, or linked thereto). 

[0057] The same data fields and principles can be applied 
to non-audio content. In video, for example, watermarked 
data can adaptively control the display monitor or playback 
parameters (e.g., color space) to enhance the viewing expe- 
rience. 

[0058] Music Asset Management/Commerce 

[0059] The majority of domestic music piracy is not 
organized. Rather, it is a crime of opportunity and conve- 
nience. If the crime were made more difficult, the alternative 
of obtaining a copy through legitimate channels would be 
less onerous. Similarly, if the procedure for obtaining a copy 
through legitimate channels were simplified, the incentive 
for piracy would be reduced. Watermarking facilitates 
both — making the crime more difficult, and making legiti- 
mate music acquisition easier. 

[0060] Consider, for example, the pricing of music in 
conventional record stores. A CD (compact disk) may cost 
$15, but its sale may be driven by just one or two popular 
songs on the disk. To obtain these songs, the consumers must 
purchase the entire disk, with perhaps a dozen songs of no 
particular interest. This, in essence, is a tying arrangement 
that benefits the record labels while prejudicing the consum- 
ers. Given these circumstances, and a ready opportunity to 
make copies, it is not surprising that customers sometimes 
make illicit copies. 

[0061] One classic technique of avoiding purchase of a 
complete collection of music, when only one or two songs 
is desired, is to record the music off the radio. While of 
dubious legality, this technique was popular in the era of 
combined cassette/radio players. However, the desired 
music was sometimes difficult to encounter in a radio 
broadcast, -and the quality was less than superb. 

[0062] The combined cassette/radio player has now 
evolved into a general purpose computer with wide-ranging 
functionality, and other sophisticated devices. Music can be 
acquired off the web, and can be recorded in various forms 
(e.g. in a personal MP3 player, stored on a hard disk, stored 
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on a wrileable CD-ROM, played back and recorded on 
analog cassette, etc., etc.). Hie quality can be quite high, and 
the erratic broadcast time problems of radio broadcasts have 
been overcome by the web's on-demand delivery mecha- 
nisms. (Moreover, the music can be downloaded in faster- 
than-realtime, a further benefit over rccording-off-the-air 
techniques.) 

[0063] One hybrid between the new and old is a novel 
radio (e.g., for use in a car) that has a "capture" button on 
the front panel (or other form of user interface, e.g., a 
Capture icon on a GUI). If a user hears a song they want to 
record and keep, they press the Capture button while the 
song is playing. In response, the radio device decodes a 
watermark embedded in the music, and thereby knows the 
identity of the music. The radio then makes a wireless 
transmission identifying the user and the desired song. A 
local repealer network picks up the wireless signal and 
relays it (e.g. by wireless rebroadcast, by modem, or other 
communication medium) to a music clearinghouse. The 
clearinghouse charges the user a nominal fee (e.g. via a 
pre-arranged credit card), and queues the music for down- 
load to a predetermined location associated with the user. 

[0064] In one embodiment, the predetermined location is 
the user's own computer. If a "live" IP address is known for 
the user's computer, the music can be transferred immedi- 
ately. If the user's computer is only occasionally connected 
to the internet, the music can be stored at a web site (e.g. 
protected with a user-set password), and can be downloaded 
to the user's computer whenever it is convenient. 

[0065] In other embodiments, the predetermined location 
is a personal music library maintained by the user. The 
library can take the form, e.g., of a hard-disk or semicon- 
ductor memory array in which the user customarily stores 
music. ITiis storage device is adapted to provide music data 
to one or more playback units employed by the user (e.g. a 
personal MP3 player, a home stereo system, a car stereo 
system, etc.). In most installations, the library is physically 
located a I the user's residence, but could be remotely sited, 
e.g. consolidated with the music libraries of many other 
users at a central location. 

[0066] The personal music library can have its own inter- 
net connection. Or it can be equipped with wireless capa- 
bilities, permitting it to receive digital music from wireless 
broadcasts (e.g. from the clearinghouse). In cither case, the 
library can provide music to the user's playback devices by 
short-range wireless broadcast. 

[0067] By such arrangement, a user can conveniently 
compile an archive of favorite music — even while away 
from home. 

[0068] Many variants of the foregoing are of course pos- 
sible. The radio can be a portable unit (e.g. a boombox, a 
Walkman radio, etc.), rather than an automotive unit. The UI 
feature employed by the user to initiate capture a musical 
selection need not be a button (physical or on-screen). For 
example, in some embodiments it can be a voice-recognition 
system that responds to spoken commands, such as "cap- 
ture" or "record." Or it can be a form of gesture interface. 

[0069] Instead of decoding the watermark only in response 
to the user's "capture" command, the radio can decode 
watermarks from all received programs, and keep the most 
recent in a small FIFO memory. By such arrangement, the 



user need not issue the capture instruction while the song is 
playing, but can do so even after the song is finished. 

[0070] In some embodiments, data corresponding to the 
watermark can be made available to the user in various 
forms. For example, it can be presented to the user on an 
LCD screen, identifying the artist and song currently play- 
ing. If a corresponding Ul button is activated, the device can 
so-identify the last several selections. Moreover, the data 
need not be presented to the user in displayed form; it can 
be annunciated by known computer-speech technologies 
instead. 

[0071] In embodiments in which the watermark docs not 
convey ASCII text data, but instead conveys ULIDs, or 
coded abbreviations, the device must generally interpret this 
data before presenting it to the user. In an illustrative 
embodiment, the device is a pocket-sized FM radio and is 
equipped with a 1 megabyte semiconductor non-volalive 
RAM memory. The memory includes a data structure that 
serves as a look-up table, matching code numbers to artist 
names and song titles. When the user queries the device to 
learn the identify of a song, the memory is indexed in 
accordance wilh one or more fields from the decoded 
watermark, and the resulting textual data from the memory 
(e.g. song title and artist) is annunciated or displayed to the 
user. 

[0072] In most applications, such memory will require 
frequent updating. The RF receiver provides a ready mecha- 
nism for providing such updated data. In one embodiment, 
the radio "awakens" briefly at otherwise idle moments and 
tunes to a predetermined frequency at which updated data 
for the memory is broadcast, cither in a baseband broadcast 
channel, or in an ancillary (e.g. SCA) channel. 

[0073] In variants of the foregoing, internet delivery of 
updated memory data can be substituted for wireless deliv- 
ery. For example, the artist/song title memory in the personal 
player can be updated by placing the player in a "nest" every 
evening. The nest (which may be integrated wilh a battery 
charger for the appliance) can have an internet connection, 
and can exchange data with the personal device by infrared, 
inductive, or other proximity-coupling technologies, or 
through metal contacts. Each evening, the nest can receive 
an updated collection of artists/song titles, and can re -write 
the memory in the personal device accordingly. By such 
arrangement, the watermark data can always be properly 
intepreted for presentation to the user. 

[0074] The "Capture" concepts noted above can be 
extended to other functions as well. One is akin to forward- 
ing of email. If a consumer hears a song that another friend 
would enjoy, the listener can send a copy of the song to the 
friend. This instruction can be issued by pressing a "Send" 
button, or by invoking a similar function on a graphical (or 
voice- or gesture-responsive) user interface. In response, the 
appliance so- instructed can query the person as to the 
recipient. The person can designate the desired recipients) 
by typing in a name, or a portion (hereof sufficient to 
uniquely identify the recipient. Or more typically, the person 
can speak the recipient's name. As is conventional wilh 
hands-free vehicle cell phones, a voice recognition unit can 
listen to the spoken instructions and identify the desired 
recipient. An "address book"-like feature has the requisite 
information for the recipient (e.g., the web site, IP address, 
or other data identifying the location to which music for that 
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recipient should stored or queued, tbe format in which the 
music should be delivered, etc.) stored therein. In response 
to such command, the appliance dispatches instructions to 
the clearinghouse, including an authorization to debit the 
sender's credit card for the music charge. Again, the clear- 
inghouse attends to delivery of the music in a desired 
manner to the specified recipient 

[0075] Still further, a listener may query the appliance (by 
voice, GUI or physical button, textual, gesture, or other 
input) to identify CDs on which the tben-playing selection is 
recorded. Or the listener may query the appliance for the 
then-playing artist's concert schedule. Again, the appliance 
can contact a remote database, relay the query, and forward 
data from the watermark pay load identifying the artist 
and/or song title to which the query relates. The database 
locates the requested data, and relays same back to the 
appliance for presentation (via a display, by machine speech, 
or other output) to the user. If desired, the user can continue 
the dialog with a further instruction, e.g., to buy one of the 
CDs on which the then-playing song is included. Again, this 
instruction may be entered by voice, GUI, etc., and dis- 
patched from (he applicance to the clearinghouse, which can 
then complete the transaction in accordance with pre-stored 
information (e.g. credit card account number, mailing 
address, etc.). A confirming message is relayed to the 
appliance for presentation to the user. 

[0076] While the foregoing transactions require a link to a 
remote site or database, other watermark-based consumer 
services can be provided without such a link. For example, 
a user can query the appliance as to the artist or song-title of 
the selection currently playing. The appliance can consult 
the embedded watermark data (and optionally consult a 
memory to determine the textual names associated with 
coded watermark data), and provide the requested informa- 
tion to the user (e.g., by a display, annunciation, or other 
output). 

[0077] The foregoing concepts (e.g. Capture, Send, etc.) 
can also be employed in connection with internet — rather 
than radio-delivery of music. (The following discussion is 
illustrated with reference to the "Capture" function, but it 
will be recognized that the other earlier-discussed features 
can be similarly implemented.) 

[0078] There are many commercial web sites that sell 
audio (in CD form or otherwise), and offer limited free 
music downloads, (or music clips) as an enticement to lure 
consumers. But there arc also a great number of music web 
sites that have no commercial pretense. They are hosted by 
music lovers strictly for the enjoyment of other music lovers. 
When music is downloaded from such a web site, the 
end-user's computer can analyze the digital data to decode 
watermark data therefrom. Again, the user can be presented 
with a "Capture" button that initiates a commercial trans- 
action, by which a complete copy of the then-downloaded 
audio is sent to a prearranged storage location, and the user's 
credit card is debited accordingly. This transaction can occur 
independently of the site from which the music is down- 
loaded (e.g. through the clearinghouse referenced above). 

[0079] While the " Capture" button can be presented on the 
web-site, this would generally not be in keeping with the 
non-commercial nature of such web sites. Instead, in an 
exemplary embodiment, the Capture feature is a software 
program resident at the user's computer. When this software 



program is invoked by the user, a socket channel is instan- 
tiated between the user's computer and the clearinghouse 
over the then-existing internet connection. The decoded 
watermark data and user ID is transmitted to the clearing- 
house over this channel, without interrupting the user's other 
activity (e.g. downloading music from the non -commercial 
web site). In response, the clearinghouse transmits the music 
to the prearranged location and attends to billing. 

[0080] In some embodiments, a watermark detector is 
included as part of the operating system, and constantly 
monitors all TCP/IP, or other internet, data received by the 
user's computer, for the presence of watermarks. In such 
case, when the Capture feature is invoked, the program 
examines a memory location in which the operating system 
stores the most-recently received watermark data. In another 
embodiment, the computer does not monitor all internet 
traffic for embedded watermark data, but includes an API 
that can be called by the Capture program to decode a 
watermark from the data then being received. The API 
returns the decoded watermark data to the Capture program, 
which relays same to the clearinghouse, as above. In still 
another embodiment, the watermark decoder forms part of 
the Capture program, which both decodes the watermark 
and relays it to the clearinghouse when the Capture program 
is invoked by the user. 

[0081] There are various techniques by which the Capture 
program can be selectively invoked. One is by a keyboard 
macro (e.g. by a combination of keyboard keys). Another is 
by a program icon that Ls always presented on the screen, and 
can be double-clicked to activate. (Again, confirmation 
processes may be called for, depending on the likelihood of 
inadvertent invocation.) Many other techniques arc likewise 
possible. 

[0082] In the just-contemplated scenario, the Capture 
operation is invoked while the user is downloading music 
from a non-commercial web site. This seems somewhat 
redundant, since the downloading — itself — is transferring 
music to the user's computer. However, the Capture opera- 
tion provides added value. 

[0083] In the case of streaming audio, the audio is not 
typically stored in a location in which it can be rc-uscd by 
the consumer. It can be listened-lo as delivered, but is then 
gone. Capturing the audio provides the user a copy that can 
be played repeatedly. 

[0084] In the case of downloaded music files, the music 
may have been encoded lo prevent its rccordal on other 
devices. Thus, while the user may download the music onto 
a desktop computer, copy-prcvention mechanisms may pre- 
vent use of that file anywhere else, e.g. on a portable music 
appliance. Again, Capturing the audio provides the user a 
copy that can be transferred to another device, fllie music 
file provided by the clearinghouse can have copy-prevention 
limits of its own — e.g., the file can be copied, but only once, 
or the file can be copied only onto devices owned by the 
user.) 

[0085J (Confirmation of device ownership can be imple- 
mented in various ways. One is to identify to the clearing- 
house all music devices owned by a user at the time the user 
registers with the clearinghouse (supplemented as necessary 
by later equipment acquisitions). Device IDs associated with 
a user can be stored in a database at the clearinghouse, and 
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Ihese can be encoded into the downloaded music as permit- 
ted devices to which the file can be copied, or on which it 
can be played.) 

[0086] The commerce opportunity presented by non-com- 
mercial music web-sites is but one enabled by digital 
watermarks. There are many others. 

[0087] To take one example, consider the media by which 
music and artists are presently promoted. In addition to radio 
airtime, these include music videos (a la MTV), fan maga- 
zines, web advertisements, graphical icons (e.g. the Grateful 
Dead dancing bears), posters, live events, movies, etc. 
Watermarked data can be used in all such media as a link in 
a commercial transaction. 

[0088] A poster, for example, typically includes a photo of 
the artist, and may comprise cover-art from a CD. The 
photo/art can be digitally watermarked with various types of 
data, e.g., the artist's identify, the record label that distrib- 
utes the artist's work, the music project being particularly 
promoted by the poster (e.g. a CD, or a concert tour), a fan 
web-site related to the artist, a web-site hosted by the record 
label for selling audio in CD or electronic form, a web-site 
from which free music by the artist can be downloaded, data 
identifying the poster itself, etc. 

[0089] A user, equipped with a portable appliance that 
merges the functions of palmtop computer and digital cam- 
era, can snap an image of the poster. The processor can 
decode the watermarked data, and initiate any of various 
links based on the decoded data. 

[0090] In an exemplary embodiment, after snapping the 
picture, the user invokes a software program on the device 
that exposes the various links gleaned from the snapped 
image data. Such a program can, for example, present the 
option of linking to the artist's fan web site, or downloading 
free streaming audio or music clips, or ordering the pro- 
moted CD, or requesting the above-noted clearinghouse to 
download a personal copy of selected song(s) by the artist to 
the user's personal music library, etc. (The device is pre- 
sumed to have a wireless internet link. In devices not having 
this capability, the requested actions can be queued and 
automatically executed when a link to the internet is avail- 
able.) 

[0091] Still more complex transactions can be realized 
with the use of a remote database indexed by digital water- 
mark fields decoded from the poster. For example, the poster 
may promote a concert tour. Fields of the digital watermark 
may identify the artist (by a code or full text), and a web site 
or IP address. The user appliance establishes a link to the 
specified site, and provides the artist identifier. In response, 
the site downloads the tour schedule for that artist, for 
display on the device. Additionally, the downloaded/dis- 
played information can include a telephone number that can 
be used to order tickets or, more directly, can indicate the 
class of seats still available at each (or a selected) venue, and 
solicit a ticket order from the user over the device. The user 
can supply requested information (e.g. mailing address and 
charge card number) over the return channel link (wireless 
or wired, as the case may be), and the ticket(s) will be 
dispatched to the user. In the case of a wireless link, all of 
this can occur while the user is standing in front of the movie 
poster. 

[0092] Similar systems can be implemented based on 
watermark data encoded in any other promotional media. 



Consider music videos. Using known TV/computer appli- 
ances, watermark data added to such videos can readily be 
decoded, and used to establish links to audio download, 
CD-sales, fan club, concert ticket outlet web sites, etc., as 
above. 

[0093] Even live events offer such watermark-based 
opportunities. The analog audio fed to public address or 
concert speakers can be watermarked (typically before 
amplification) to encode plural-bit digital data therein. A 
next generation personal music appliance (e.g. one with a 
wireless interface to the internet) can include analog record 
capability (e.g. a built-in microphone, analog-to-digital con- 
verter, MP3 encoder, coupled to the unit's semiconductor 
memory). A user who attends a live event may record an 
excerpt of the music. The watermark can then be decoded, 
and the extracted data used to access the links and commerce 
opportunities reviewed above. 

[0094] Cinema movies offer both audio and visual oppor- 
tunities for watermark-based commerce opportunities. 
Either medium can be encoded to convey information of the 
types reviewed above. A personal appliance with image- or 
audio-capture capabilities can capture an excerpt of the 
audio or imagery, decode the watermark data therefrom, and 
perform any of the linking, etc., functions reviewed above. 

[0095] The consumer-interest watermarks reviewed above 
are only exemplary. Many others will be recognized as 
useful. For example, promotional clips presented before a 
feature film presentation can include watermark data that 
point (either by a literally encoded web address link, or by 
an ID code that indexes a literal link in a remote link 
database) to reviewer critiques of the previewed movies. 
Watermark data in a featured film presentation can lead to 
web sites with information about the movie stars, the direc- 
tor, the producer, and can list other movies by each of these 
persons. Other watermark -conveyed web links can present 
opportunities to buy the movie on videotape, to purchase the 
movie soundtrack, to buy movie-related toys and games, etc. 

[0096] More on Device Control 

[0097] Much of the foregoing has focused on watermark 
encoding to provide enhanced customer experiences or 
opportunities. Naturally, watermarks data can alternatively, 
or additionally, serve the interests of the media owner. 

[0098] To illustrate, consider watermarked music. The 
media owner would be best served if the watermark serves 
dual purposes: permissive and restrictive. Permissively, 
music appliances can be designed to play (or record) only 
music that includes an embedded watermark signaling that 
such activity is authorized. By this arrangement, if masic is 
obtained from an unauthorized source and does not include 
the necessary watermark, the appliance will recognize that it 
does not have permission to use the music, so will refuse 
requests to play (or record). 

[0099] As noted, music appliances can respond restric- 
tive ly to th$ embedded watermark data to set limits on use 
of the music. Fields in the watermark can specify any or all 
of (or others in addition to) (a) the types of devices on which 
the music can be played (b) the types of devices on which 
the music can be recorded; (c) the number of times the music 
can be played; (d) the number of times the music can be 
recorded, etc. 
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[0100] The device restrictions (a) and (b) can be of various 
types. In some embodiments, the restrictions can identify 
partial la r units (e.g. by serial number, registered owner, etc.) 
that are authorized lo play/record the encoded music. Or the 
restrictions can identify particular classes of units (e.g., 
battery-powered portable players with music memories of 
less than 50 megabytes, disk-based dedicated music appli- 
ances, general purpose personal computers, etc.) Or the 
restrictions can identify particular performance quality cri- 
teria (e.g., two channel, 16-bit audio at 44.1 KHz sample 
rate, or lower quality). 

[0101] The use restrict ions (c) and (d) can likewise be of 
various types. Examples include "do not copy,""copy once 
only, ""unrestricted copying permitted,"" play once,""play N 
limes" (where N is a parameter specified elsewhere in the 
watermarked data, or by reference to a database indexed by 
a watermark data field), "unrestricted playing permitted," 
etc. 

[0102] It is straightforward to design a music appliance to 
respond to usage limits of zero (e.g. "do not copy") and 
infinity (e.g. "unrestricted copying permitted," and "unre- 
stricted playing permitted"). The device simply examines 
one or more bits in the watermark data, and permits (or 
refuses) an operation based on the value thereof. 

[0103] Implementation of the other usage-control restric- 
tions can proceed in various ways. Generally speaking, the 
stored music can be altered to give effect to the usage - 
control restrictions. For example, if the music Ls "record - 
once," then at the time of recording, the appliance can alter 
the music in a fashion indicating that it now has "do not 
record" status. This alteration can be done, e.g., by changing 
the watermark data embedded in the stored music (or adding 
watermark data), by changing other data stored in associa- 
tion with (he music, etc. If the original signal is stored (as 
opposed, e.g., to a streaming signal, such as an internet or 
wireless transmission), it too should be so-altered. 

[0104] Likewise with playback limitations. The number of 
playbacks remaining can, e.g., be encoded in an updated 
watermark in the music, be tracked in a separate counter, etc. 

[0105] More particularly considering the "copy once" 
usage restriction, an illustrative embodiment provides two 
distinct watermark pay load bits: a "copy once" bit and a 
•'copy never bit. When originally distributed (whether by 
internet, wireless, or otherwise), the "copy once" bit is set, 
and the "copy never" bit is un-set. 

[0106] When music encoded in this fashion is provided to 
a compliant recording device, the device is authorized to 
make one copy. (A compliant device is one that recognizes 
encoded watermark data, and behaves as dictated by the 
watermark.) When this privilege is exercised, the recording 
device must alter the data to ensure that no further copying 
is possible. In the illustrated embodiment, this alteration is 
effected by the recording device adding a second watermark 
to both the music, with the "copy never" bit asserted. The 
second watermark must generally be encoded in an 
"orthogonal" domain, so that il will be delectable notwith- 
standing the continued presence of the original watermark. 
Compliant equipment must then check for both watermarks, 
and refuse lo copy if cither is found to have the "copy never" 
bit asserted. 

[0107] One advantage to this arrangement is that if the 
watermark signal has undergone some form of corruption 



(e.g. scaling or resampling), the first watermark may have 
been weakened. In contrast, the second watermark will be 
native to the corrupted signal, and thus be more easily 
detected. (The corruption may aLso contribute to the 
orthogonality of one watermark relative to the other, since 
the two watermarks may not have precisely the same time 
base or other foundation.) 

[0108] An alternative approach is not lo encode the "copy 
never" bit in the original music, but leave this bit (in 
whatever manifestation) blank (i.e. neither "1" nor "0"). In 
transform-based watermark techniques, this can mean leav- 
ing transform coefficienl(s) corresponding to the "copy 
never" bit unchanged. If the watermarking Ls effected in the 
temporal sample domain (or spatial domain, for image data), 
this can mean leaving certain samples (pixels) unmodified. 
The recording device can then alter the transform coeffi- 
cients and/or samples as necessary to assert the previously- 
unencoded "copy never" bit when the permitted recording is 
made. 

[0109] In such a system, compliant recording devices 
check for the "copy never" bit in the sole watermark, and 
refuse to make a copy if it is asserted (ignoring the value of 
any "copy once" bit). 

[0110] A third approach to "copy once" Ls to set both the 
"copy once" and "copy never" bits, but set the former bit 
very weakly (e.g. using lower gain and/or high frequency 
DCT coefficients that do not survive certain processing). The 
frail "copy once" bit is designed not to survive common 
corruptions, e.g., resampling, scaling, digital to analog con- 
version, etc. To further assure that the "copy once" bit is lost, 
the recording device can deliberately add a weak noise 
signal that masks this bit (e.g. by adding a noise signal in the 
frequency band whose DCT coefficient conveys the "copy 
once" bit). In contrast, the "never copy" bit is unchanged and 
reliably detectable. 

[01 U] In such a system, compliant devices check for the 
"copy once" bit in the sole watermark, and refuse to make 
a copy if it is not detected as set. 

[0112] These three examples are but illustrations of many 
possible techniques for changing the rights associated with 
a work. Many other techniques are known. See, e.g., I lie 
proposals for watermark-based copy control systems for 
digital video at the Copy Protection Technical Working 
Group, http://www.dvcc.com/dhsg/, from which certain of 
the foregoing examples are drawn. See also Bloom el al, 
"Copy Protection for DVD Video," IEEE Proceedings, 
Special Issue on Identification and Protection of Multimedia 
Information, June, 1999. 

[0113] Scalcability 

[0114] One feature that is desirable in many detector 
embodiments is scaleability. This refers to the ability of a 
detector to scale its computational demands lo match the 
computational resources available to it. If a detector is 
running on a high performance Pentium III workstation, il 
should pe 'Adoing more" than if the same detector is running 
on a slow microcontroller. One way scalability can be 
achieved is by processing more or less chunks of input data 
(e.g. temporal excerpts of music, or blocks/macroblocks of 
pixels in a frame of video data) to decode watermarks. For 
example, an input audio stream might be broken into chunks 
of one second each. A fast processor may complete decoding 
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of each ciiunk in less than a second, permitting il succes- 
sively to process each chunk in the data stream. In contrast, 
a slow processor may require two and a half seconds to 
decode the watermark from a chunk. While it is processing 
a first chunk, the second and third pass by un-decoded. The 
processor next grabs and processes the fourth chunk, per- 
mitting the fifth and sixth to pass by un-encoded. 

[0115] The detector running on the fast processor is clearly 
more difficult to "fool," and yields a decoded watermark of 
higher confidence. Bui both systems decode the watermark, 
ami both operate in "real lime." 

[0116] The skipping of input data in the temporal (e.g. 
music or video) or spatial (e.g. image or video) domain is but 
one example of how scale ability can be achieved. Many 
other approaches are known to those skilled in the art. Some 
of these alternatives rely on spending more or less time in 
the data analysis phases of watermark decoding, such as 
cross-correlation operations. 

[0117] Reference has been made to watermarked UIDs as 
referring to a database from which larger data strings (e.g. 
web addresses, musician names, etc.) can be retrieved. In 
some embodiments, the data record referenced by a UID 
can, in turn, point to several other database records. By such 
arrangements, it is often possible to reduce the payioad of 
the watermark, since a single UID reference can lead to 
several different data records. 

[0118] Production Tools 

[0119] In the prior art, the watermark embedded in a 
source material is typically consistent and sialic through a 
work — unchanging from beginning to end. But as will be 
recognized from the foregoing, there are many applications 
that are belter served by changing the watermark data 
dynamically during the course of the work. According to 
another aspect of the invention, a production tool is provided 
that facilitates the selection and embedding of dynamically- 
changing watermark data. One such embodiment is a soft- 
ware program having a user interface that graphically dis- 
plays the different watermark fields that are being embedded 
in a work, and presents a library of data (textually or by 
icons) that can be inserted into each field, and/or permits the 
user to type in data to be encoded. Another control on the UI 
controls the advance and rewind of the media, permitting the 
user to determine the location at which different watermark 
data begins and ends. Graphical paradigms known from 
video- and audio-editing tools can be used to indicate the 
starting and ending frames/samples for each different water- 
mark payioad. 

[0120] Such a tool can be of the standalone variety, or can 
be integrated into the desktop audio- and video-production 
and editing tools offered by vendors such as Avid, Adobe, 
Jaleo, Pinnacle Systems, Sound Forge, Sonic Foundry, Xing 
Technology, Prosoniq, and Sonic Desktop Software. 

[0121] Payment-based Systems 

[0122] Another aspect of the present invention is the use 
of anonymous payment tokeas that can be used to obtain 
content on the web. In one embodiment, a token comprises 
a 128-bil pseudo-random number, to which additional bits 
identifying an issuing bank (or other issuing institution) are 
appended. (The additional bits can be the IPaddrcssof a web 
server of the bank, a routing number identifying the bank for 



electronic wire transfers, or other identifier.) The 128-bil 
numbers are randomly generated by the bank — commonly 
as needed — and each represents a fixed increment of money, 
e.g. ten cents. 

[0123] A consumer wishing to have a store of currency for 
such commerce pays the bank, e.g., S10 in exchange for 100 
tokens. These tokens are transferred electronically to disk or 
other storage in the consumer's computer in response, e.g., 
to a credit card authorization, or may be provided by diskette 
or other storage medium over the counter at a bank branch 
(in which case the consumer thereafter copies the numbers 
into storage of his or her computer). (Outlets other than 
banks can of course be employed for distributing such 
numbers, much in the manner that convenience and many 
grocery stores commonly issue money orders.) 

[0124] Imagine that the consumer wishes lo view the final 
quarter of a Trailblazer basketball game that aired on tele- 
vision a week ago. (The consumer may have either missed 
the game, or may have seen it but wants to see the last 
quarter again.) The user directs a web browser to a web site 
maintained for such purpose and performs a search to 
identify the desired program. (Typically, the web site is 
maintained by the proprietor that holds the copyright in the 
material, but this need not be the case. Some material may 
be available at several web sites, e.g., maintained by ABC 
Sports, the National Basketball Association, and Sports 
Illustrated.) The search can use any of various known search 
engines, e.g., Infoscek, Verity, etc., and can permit searching 
by title terms, keywords, date of airing, copyright owner, etc. 
By typing in, e.g., the keyword * Trailblazers' and the date 
'4/26/99,' the consumer is presented a listing of videos 
available for download. One, hopefully, is the requested 
game. With each listing is an indication of an associated 
nominal charge (e.g. SO cents). 

[0125] On clicking on a hypertext link associated with the 
desired basketball game, the viewer is presented a further 
screen with one or more options. ITie first of the listed 
options is the entire game, with commercials. The charge is 
the nominal charge presented on the earlier screen (i.e. 80 
cents). Other options may include the first, second, third, and 
fourth quarters of the game individually, each of which — 
save the last, costs 20 cents. The last may be charged at a 
premium rate, e.g., 30 cents. Clicking on the desired video 
option yields a further screen through which payment is 
effected. 

[0126] To pay for the requested video, the consumer 
instructs his or her computer to transfer three of the earlier- 
purchased tokens over the web to the video provider. Various 
user interface metaphors can be employed lo facilitate this 
transfer, e.g., permitting the user to type the amount of 
money to be transferred in a dialog box presented on-screen, 
or dropping/dragging icons representing tokens from an 
on-screen "wallet" to an on-screen "ticket booth" (or over an 
icon or thumbnail representing the desired content), clicking 
on an "increment" counter displayed adjacent the lisling of 
the coqtenh etc. Once the consumer has authorized a transfer 
of sufficient tokens, the coasumer's computer sends to the 
web site (or lo such other web address as HTML encoding 
in the viewed web page may indicate) the tokens. This 
transmission simply lakes the form of the three 128+bit 
numbers (the V indicating the bank identifier) — in what- 
ever packet or other format may be used by the internet link. 
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Once dispatched in this manner, the tokens are deleted from 
the user's computer, or simply marked as spent. (Of course, 
in other embodiments, a record of the expenditure may be 
stored in the consumer's computer, e.g., with the token 
contents and a record of the audio or video purchase to 
which they were applied.) 

[0127] Since the amount of money is nominal, no encryp- 
tion is provided in this embodiment, although encryption 
can naturally be provided in other embodiments (e.g., either 
in sending the tokens from the user to the web site, or earlier, 
in sending the tokens to the user). As will be seen, provided 
that the media provider immediately sends the tokens to the 
bank in real time, encryption is a nice feature but not 
mandatory 

[0128] On receipt of the token data, the web site imme- 
diately routes the token data to the identified bank, together 
with an identifier of the media provider or account to which 
the funds represented thereby are to be credited. The bank 
checks whether the 128 -bit numbers have been issued by 
that bank, and whether they have already been spent. If the 
numbers are valid, the bank updates its disk-based records to 
indicate that the three tokens have been spent and that the 
bank now owes the media supplier 30 cents, which it may 
either pay immediately (e.g., by crediting to an account 
identified by the media provider) or as one lump sum at the 
end of the month. The bank then sends a message to the web 
site confirming that the tokens were valid and credited to the 
requested account. (Optionally, a message can be sent to the 
purchaser of the tokens (if known), reporting that the tokens 
have been redeemed.) 

[0129] In response, the web site begins delivery of the 
requested video to the consumer. In the illustrated embodi- 
ment, the video is watermarked prior to delivery, but oth- 
erwise sent in unencrypted fashion, typically in streaming 
format, but optionally in file format. (Encryption can be used 
in other embodiments.) The watermarking in the illustrated 
embodiment is accomplished on -the -fly and can include 
various data, including the date of downloading, the down- 
load site, the destination IP address, the identity of the 
purchaser (if known), etc. 

[0130] The large size of the video and the small charge 
assessed therefor provide disincentives for the consumer 
making illicit copies. (Especially as to archival material 
whose value decays with time, there is not much after- 
market demand that could be served by illicit copies, making 
third party compilation of such material for re-distribution 
financially unattractive. First run video, and material that 
keeps a high value over time, would not be as well suited for 
such distribution, and could better employ technology dis- 
closed elsewhere herein.) 

[0131] In some embodiments, the integrity of the received 
video is checked on receipt. This feature is described below 
in the section entitled Watermark-Based Receipts. 

[0132] In the illustrative system, nothing in the tokens 
indicates the identity of the purchaser. The web site knows 
the IP address of the site to which video was delivered, but 
need not otherwise know the identity of the purchaser. The 
bank would probably maintain a record of who purchased 
the tokens, but need not. In any event, such tokens could 
thereafter be exchanged among consumers, resulting in 
anonymity from the bank, if desired. 



[0133] As described above, the video excerpts from which 
the consumer can select include commercials. At some sites, 
video may be provided from which the commercials have 
been excised, or which is delivered in a manner that skips 
past the commercials without transmitting same to the 
consumer. Such video will naturally command a premium 
price. In some embodiments, the difference in price is 
electronically credited as compensation to accounts main- 
tained for (or by) the advertisers, whose advertisements are 
not being viewed by such consumers. (The identification of 
advertisers to be credited is desirably permanently encoded 
in the video, either throughout the video (if the video has had 
the commercials removed therefrom), or by data in the 
commercials themselves (which commercials are skipped 
for transmission to the consumer, but can still be decoded at 
the video head-end. Such encoding can be by in-band 
watermarking or otherwise.) 

[0134] While the foregoing discussion particularly con- 
sidered video as the desired content, the same principles are 
equally applicable in connection with audio, still imagery, 
and other content. 

[0135] The token-based payment method is but one of 
many that can be employed; the literature relating to on-line 
payment mechanisms is extensive, and all such systems can 
generally be here-employed. 

[0136] Tracking 1 28-bit tokens can be a logistical problem 
for the bank. One approach is to have a memory with 10 12S 
locations, and at each location store a two-bit value (e.g. 
00«never issued; 01«issued but not spent; 10- issued and 
spent; 11 -reserved). More complete data could alternatively 
be stored, but such a memory would be unpractically large. 

[0137] One alternative approach is to hash each 128-bit 
number, when issued, to a much smaller key value (e.g. 20 
bits). A memory with 10 20 locations can be indexed by this 
key. Each such location can include four data: an issued 
128-bit token number that hashes to that value, first and 
second date fields indicating the date/time on which that 
token was issued and redeemed, respectively, and a link 
specifying the address of a next memory location. That next 
memory location (outside of the original 10?° locations) can 
include four more data, this time for a second issued-1 28-bit 
token number that hashed to the original key value, two date 
fields, and again with a link to a subsequent storage location, 
etc. 

[0138] When a 128 -bit random number is generated, the 
original memory location indexed by the hash code of that 
number is checked for an earlier number of the identical 
value (to avoid issuance of duplicate tokens). Each succes- 
sive location in the linked chain of memory locations is 
checked for the same 128-bit number. When the end of the 
linked chain is reached, the bank knows that the 128-bit 
random number has not previously been issued, and writes 
that number in the last-addressed location, together with the 
date of issuance, and a link to a next storage location. 

[0139] >yhen a 128 -bit token is received, the same linked- 
list processing occurs to identify a first location, and to 
thereafter step through each subsequent location until a 
match is found between the token number and the number 
stored in one of the linked memory locations. When found, 
that number is marked as redeemed by writing a redemption 
date/time in the corresponding field. If the search reaches the 
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end of the linked chain without finding a match between the 
stored numbers and the token number, the token is treated as 
invalid (i.e. not issued by that bank). 

[0140] Other manners of tracking the large number of 
possible token numbers can of course be used; the foregoing 
is just exemplary. Or the tokens needn't be tracked at all. 
Such an arrangement is highly practical if the token has 
sufficient bits. With the illustrated 128 bits, for example, the 
chance of two identical tokens being issued Ls infinitesimally 
small, so checking for duplicate issuance can be omitted if 
desired. In such case, the bank can simply maintain an 
ordered list of the token numbers still outstanding and valid. 
As new tokens arc dispensed, their token numbers are added 
to the list. As tokens are redeemed, their numbers arc deleted 
from the list. Known list processing techniques can be 
employed to speed such search, update, and delete actions. 

[0141] Watermark-based Receipts 

[0142] Pay-for-conlent applications commonly assume 
that if content is transmitted from a server (or head-end, 
etc.), it is necessarily received. Sometimes this assumption 
is wrong. Network outages and interruptions and internet 
traffic load can diminish (e.g., dropped video frames), or 
even negate (e.g., failed delivery), expected consumer 
enjoyment of content. In such cases, the consumer is left to 
haggle with the content provider in order to obtain an 
adjustment, or refund, of assessed charges. 

[0143] Watermarks provide a mechanism for confirming 
receipt of content. If a watermark is detected continuously 
during a download or other delivery event, a software 
program (or hardware device) can issue an electronic receipt 
attesting that the content was properly delivered. This 
receipt can be stored, and/or sent to the content distributor 
to confirm delivery. 

[0144] In one embodiment, a content receiving device 
(e.g., computer, television or set-top box, audio appliance, 
etc.) periodically decodes a watermark from the received 
content to confirm its continued reception. For example, 
every five seconds a watermark detector can decode the 
watermark and make a record of the decoded data (or simply 
record the fact of continued detection of the same water- 
mark). When a changed watermark is detected (i.e., recep- 
tion of a diflferenl content object begins), the duration of the 
previously-received content is logged, and a receipt is 
issued. 

[0145] In a related embodiment, the last portion (e.g., 5 
seconds, frame, etc.) of the content bears a different "end of 
content" watermark that triggers issuance of a receipt. Such 
a watermark can indicate the length of the content, to serve 
as a cross-check against the periodic watermark polling. 
(E.g., if periodic sampling at 2 second intervals yields 545 
samples corresponding to the same content, and if the "end 
of content" watermark indicates that the content was 1090 
seconds long, then receipt of the entire content can be 
confirmed.) 

[0146] In another embodiment, the watermark can change 
during the course of the content by including, e.g., a datum 
that increments every frame or other increment of time (e.g., 
frame number, lime stamp, etc.). A watermark detector can 
monitor the continued incrementing of this datum through- 
out the content to confirm that no part was garbled (which 
would destroy the watermark) or was otherwise missing. 



Again, at the end of delivery, the receiving system can issue 
a confirmation that XXX frames/scconds/ete. of the identi- 
fied content were received. 

[0147] One application of such technology Ls to bill for 
content based on receipt, rather than transmission. More- 
over, billings can be adjusted based on jwrcentage of con- 
tent-value received. If delivery is interrupted mid-way 
through (e.g., by the consumer disabling the content -receiv- 
ing device), the nominal billing for the content can be 
halved. Some prolonged content, e.g., lelevised/web-broad- 
cast university classes, cannot be "consumed" in one ses- 
sion, and are thus particularly well suited for such pay-as- 
you-consumc billing. 

[0148] Another application of such technology is in adver- 
tising verification. Presently, ads are tracked by transmission 
or, less frequently, by detection of an embedded code on 
receipt (c.f., Nielsen Media Research's U.S. Pat. Nos. 5,850, 
249 and 5,737,025). However, such reception-detectors — 
once triggered — generally do not further note the length of 
time that the advertising was received, so the same data is 
produced regardless of whether only five or fifty seconds of 
a commercial is presented. Watermark monitoring as con- 
templated herein allows the duration of the advertising 
impression to be precisely tracked. 

[0149] In one application of this technology, recipients of 
advertising arc provided incentives for viewing advertising 
in its entirety. For example, a content-receiving device can 
include a watermark detector that issues a receipt for each 
advertisement that is heard/viewed in its entirety. These 
receipts can be redeemed, e.g., for content tokens as 
described elsewhere herein, for monetary value, etc. In some 
embodiments, receipts arc generic and can all be applied to 
a desired premium, regardless of the advertisements through 
which they were earned. In other embodiments, the receipts 
are associated with the particular advertisers (or class of 
advertisers). Thus, a TV viewer who accumulates 50 receipts 
from advertising originating from Procter & Gamble may be 
able to redeem same for a coupon good for S2.50 oil any 
Procter & Gamble product, or receipts from Delta Airlines 
may be redeemed for Delta frequency llier miles (e.g., at a 
rate of one mile per minute of advertising). Such incentives 
are particularly useful in new forms of media that give the 
consumer enhanced opportunities to fast -forward or other- 
wise skip advertising. 

[0150] (Although the foregoing "receipt" concept has 
been described in conjunction with watermark data (and use 
of watermark technology is believed to be inherently advan- 
tageous in this application), the same principles can likewise 
be implemented with ancillary data conveyed by other 
means.) 

[0151] Master Global Address 

[0152] As suggested above, it is desirable that each piece 
of content have a web address (the "Master Global Address" 
(MGA), or "Master IP Address") associated with it. Such 
address is typically conveyed with the content, e.g., by an IP 
address watermarked therein, 

[0153] Consider a consumer who downloads a streaming 
video having an English language soundtrack. The viewer 
may not speak English, or may otherwise prefer to listen to 
the soundtrack in another language. The user can decode the 
watermark data embedded in the video and initiate a link to 
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the associated web address. There the user is presented with 
a list of soundtracks for that content object in other lan- 
guages. The viewer can click on the desired language and 
receive same via a second simultaneous transmission (e.g., 
a second socket channel). The consumer's audio/video 
appliance can substitute the desired audio track for the 
default English track. 

[0154] If the streaming video and the alternative 
soundtrack are hosted on the same server, synchronization is 
straightforward. The process governing transmission of the 
alternative soundtrack identifies the process that is streaming 
video to the same IP address. Based on SMPTE, or other 
time/frame data, the former process syncs to the latter. (If the 
two data streams don't originate through the same server, 
time/frame data can be relayed as necessary to the alterna- 
tive soundtrack server to effect synchronization.) 

[0155] Another application of the Master Global Address 
is to serve as a point to which monitoring stations can report 
the presence, or passage, of content. Consider, for example, 
a copyright-aware node through which content signals pass, 
e.g., a computer node on a network, a satellite transponder, 
etc. Whenever the node detects passage of a media object 
(e.g., by reference to a file extension, such as MP3, JPG, 
AVI, etc.), it sends a "ping" over the internet to the address 
encoded in the object, simply reporting passage of the 
object. Similar monitoring facilities can be provided in end 
user computers, e.g., reporting FileOpen, FileSave, Printing, 
or other use of content bearing MGA data. 

[0156] This system can be expanded to include "ping" and 
"pong" phases of operation. When a software application (or 
a user appliance, such as a video or audio playback device) 
encounters a media object (e.g., at time of file open, at time 
of playback, etc.), it pings the MGA site to report the 
encounter. The MGA site "pongs" back, responding with 
instructions appropriate to the encounter. For example, if the 
object requires payment of a fee before full functionality or 
access is to be granted, the MGA site can respond to the 
application with instructions that the object be used (e.g., 
played back) only in some crippled state preventing the 
user's full enjoyment (e.g., impaired resolution, or impaired 
sound quality, or excerpts only, etc.). The MGA site can also 
inform the user application of the terms (e.g., payment) by 
which full functionality can be obtained. The application can 
graphically or audibly present such information to the user, 
who can authorize a payment, if desired, so that the content 
can be enjoyed in a less- (or un-) crippled state. On receipt 
of the payment authorization, the MGA site can inform the 
user application that enhanced access/usage rights have been 
purchased, and that the application may proceed accord- 
ingly. 

[0157] Yet another application of the MGA is to present 
the user of a content object a menu of options that is 
customized to that object. 

[0158] In current graphical operating systems, when a user 
clicks on an icon (e.g., with the right mouse button), a menu 
is presented detailing actions that can be undertaken in 
connection with the icon, or the file represented thereby. 
Such options are pre-programmed (i.e., static), and are 
typically determined by the operating system based solely 
on the file extension. 

[0159] In accordance with this aspect of the present inven- 
tion, clicking on an icon representing a media object initiates 



an internet link to the MGA site associated with the object. 
The MGA site responds with data that is used to customize 
the menu of options presented to the user in connection with 
that particular object. 

[0160] Consider an icon representing a JPG image file. 
Right-clicking on the icon may yield a menu that gives the 
user various options presented by the operating system (e.g., 
delete, compress, rename), and additional options custom- 
ized in accordance with data from the object's MGA site. 
These customized options may include, e.g., 

[0161] (a) open in 100x150 pixel format for free; 

[0162] (b) open in 480x640 pixel format for ten 
cents; 

[0163] (c) open in 960x1280 pixel format for twenty 
cents; 

[0164] (d) purchase rights to use this image in a 
newsletter having a circulation of under 1000 for 
$1.25; 

[0165] (e) display a complete listing of license 
options. 

[0166] Clicking on options (b) or (c) initiates a commerce 
application through which funds are electronically trans- 
ferred to the MGA site (by the above-described tokens or 
otherwise). In response, the MGA site responds (e.g., with 
TCP/IP or HTML instructions) authorizing an application on 
the user's computer to open the file in the requested manner. 
(The default application for JPG applications can then 
automatically be launched, or the computer may first query 
the user whether another application should be used 
instead.) 

[0167] Clicking on option (d) proceeds as above, and 
permits full use of the image on the computer. Moreover, the 
MGA site sends a digital certificate to the user's computer 
memorializing the usage rights purchased by the consumer. 

[0168] In this particular arrangement, no access control is 
placed on the content, e.g., by encryption, secure container 
technology, or the like. The nominal fees, and the ease of 
licensing, make it simple for the user to "do the right thing" 
and avoid copyright liability. In other embodiments, of 
course, known access control techniques can be used to limit 
use of the object until the requisite payment has been made. 

[0169] Naturally, records of all such transactions are also 
logged at the MGA site. 

[0170] Clicking on option (e) opens a browser window on 
the user's computer to a web site that presents a complete 
listing of license options available for that image. (The 
address of this web site is included in customization data 
relayed to the user device from the MGA site, but not 
explicitly shown to the user on the menu.) Through such web 
site, the user can select desired rights, effect payment, and 
receive the necessary authorization for software applications 
on the user's computer (or other media appliance) to open 
and/or process the content. 

[0171] The object on which the user "clicks" needn't be an 
icon. It can be an image or other graphical representation. 
(And a "click" isn't necessary; a voice command or other 
signal may be used to the same effect with an audio clip or 
selection.) 
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[0172] Consider the popular merchandising of books and 
CDs over the internet. A JPG or other image file depicting 
the cover of a book, or the artwork of a CD cover, can be 
treated as a media object, and can include a watermarked 
MGA pointer. Right-clicking on such an image of a book 
cover could, through the MGA site, present to the user a 
menu of options that includes — in addition to those normally 
presented in conjunction with a JPG file — the following: 

[0173] (a) "See the review of this book published in 
the New York Times on Apr. 19, 1999" 

[0174] (b) "See the list of reviews of this book at 
Amazon.com" 

[0175] (c) "Enter your own review of this book, for 
posting on Amazon.com" 

[0176] (d) "See today's sales rank of this book at 
Amazon.com" 

[0177] (e) "Purchase this book from Amazon.com for 
$16.95" 

[0178] (f) "Purchase this book from Bamesandnoble- 
.com for $19.95 and receive a $5.00 credit towards 
your next purchase" 

[0179] (g) "Link to the web site that tells about the 
release of this title as a motion picture (presently 
scheduled to open on Oct. 10, 1999)" 

[0180] (h) "Link to the Yahoo listing of web sites 
relating to this book" 

[0181] (i) "Search Lycos for listings relating to this 
book." 

[0182] If the user selects one of the purchase options from 
the menu, a pre-stored e-commerce profile — containing the 
user name, credit card number, billing address, ship-to 
address, etc., possibly in the form of an encrypted object — 
could be sent to the MGA site (or to the bookseller) to effect 
the purchase, or such selection could initiate display of 
additional screens or sub-menus through which the user 
would manually enter or select such information for trans- 
mission. 

[0183] Others of the selections cause a new browser 
window to open on the user's computer, opening to a URL 
specified in data relayed from the MGA site but not dis- 
played to the user in the menu. Appropriate HTML instruc- 
tions can be generated to effect a particular query or other 
operation at the specified URL. 

[0184] In some embodiments, the customized menu pre- 
sents only a single choice in addition to those normally 
provided by the operating system, e.g., "Link to home." 
Clicking on this option opens a browser window to a home 
page at the MGA for that object. On that page, the user is 
presented with all of the foregoing options, and more 
(possibly including advertising graphics or multi -media). 
Such objects can serve as powerful marketing agents. 
Returning to the example discussed above, a JPG image file 
of a book cover may have, as its MGA, a web page hosted 
by a particular bookseller, providing purchase options and 
other information for that book. Marketing of books (or 
CDs, or cars, or consumer appliances, or virtually anything 
else) can be effected by disseminating such vendor-issued 
JPGs as widely as possible. Some book cover JPGs may be 



distributed by Amazon.com, others by Barnes&Nob le.com, 
others by Borders.com — each pointing back to a different 
MGA through which purchase transactions for that book 
may be performed. 

[0185] Returning to the MGA-customized menus, these 
needn't be limited to menus resulting from clicking on an 
icon or image (or signaling during an audio excerpt). Drop- 
down menus in application programs can likewise be popu- 
lated with customized options, in accordance with customi- 
zation data obtained from the MGA site for the object 
presently being accessed or used. Most graphical operating 
systems and application programs have well developed 
toolsets permitting such menu customization. Again, other 
data relayed from the MGA site is not shown to the user, but 
is employed by the computer (e.g., a browser program) to 
carry out menu options selected by the user. 

[0186] Again the foregoing techniques are equally appli- 
cable for still images, audio, video, and other forms of 
content, and can readily be adapted for use both with general 
purpose computers, software applications, and specialized 
media appliances. 

[0187] While, for expository convenience, the foregoing 
discussion contemplated embedding a literal URL address in 
the object as the MGA, more typically this is not the case. 
Instead, the MGA more commonly comprises identification 
data for the object (e.g. a 128-bit random ID), together with 
the URL for a name server computer that serves many 
(perhaps millions) of such objects (an example of the latter 
is the Digimare MarcCentre server). 

[0188] To obtain the desired data as detailed above, the 
user's computer (sometimes termed a client computer) links 
to the name server computer and provides the ID of the 
object being processed. The name server computer uses this 
ID to query a database, and obtains from the database the 
current IP address to which such queries should be routed. 
The name server computer can relay the request from the 
client computer to the correct destination address, or can 
return the correct destination address to the client computer, 
which can initiate such a link itself. By such arrangement, 
the IP address ultimately associated with an object can be 
easily changed as needed, simply by changing the corre- 
sponding record in the name server database, without ren- 
dering obsolete legacy objects having out-of-date addresses 
encoded therein. 

[0189] In some embodiments, the URL of the name server 
needn't be included in the watermark. In the absence of a 
specified URL, the client computer may direct such links to 
a default name server address instead (stored locally or 
remotely). If that server doesn't recognize the object ID, it 
can return an error code, or pass the query on to other name 
servers. Those servers, in turn, can pass the query along to 
still other name servers if they don't recognize the object ID. 
In this fashion, an exponentially-large number of name 
servers might be quickly polled for information relating to 
the identified object. Alternatively, rather than encoding the 
complete'lP address of the name server in an object water- 
mark, the first N (e.g., 16) bits of the object ID might be used 
as a short-hand for one of 65,536 predetermined name server 
addresses, in accordance with data stored locally (e.g., on 
RAM or disk in the user's computer) or remotely (e.g., at a 
default name server IP address). 
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[0190] While the basic concept idea behind embedding 
MGAdata within an object is to point to a repository of data 
about the object, a pointer the other way may be achieved as 
well. 

[0191] As noted, the "ping" application of MGA data 
permits an MGA site to be informed of sites through which 
its object passes. More generally, the MGA site can log the 
originating address of each query it receives. Each such 
address can be presumed to have (or have had) a copy of the 
corresponding object. Media owners can thereby track the 
dissemination of copies of their media objects — at least 
insofar as use of such objects entails communicating with 
the associated MGA site. 

[0192] Such tracking offers a great number of opportuni- 
ties, some in the area of commerce. The MGA site corre- 
sponding to the cover art of a Garth Brooks CD, for 
example, can provide a listing of IP addresses of persons 
interested in that CD. Email or promotional data objects 
(e.g., audio clips) can be sent to that list of addresses when 
a subsequent Garth Brooks CD is released. 

[0193] Such tracking also opens up a new dimension of 
internet searching. Presently, internet search engines use a 
brute force approach, visiting millions of pages across the 
web in order to identify, for example, a dozen instances of 
a given photograph file. MG As offer a shortcut to such brute 
force approaches. With the present technology, a search 
engine can find a single instance of a photograph file and, by 
detection of the MGA data watermarked therein, link to the 
corresponding MGA site. From the MGA site, the search 
engine can obtain a listing (if such queries are authorized) of 
some or all of the other sites known by the MGA site to have 
copies of that photograph file. (Providing such data to search 
engines is a commerce opportunity for such MGA sites, 
which may permit such access to its listing of sites only in 
exchange for a fee. Or the MGA site may arrange to collect 
a tribute payment from the search engine proprietor each 
time the engine responds to a user query using data collected 
from the MGA site.) 

[0194] Many of the addresses logged by the MGA may not 
be publicly-accessible data stores. The search engine can 
check each listed address to ensure that the desired object is 
present and accessible before adding the address to its 
database. 

[0195] Covert Tracing 

[0196] Co-pending application Ser. No. 09/185,380 
describes anti-counterfeit ing technology that looks for the 
presence of digital data corresponding to bank note imagery 
in a computer system, and makes a covert record of any 
attempt to process such data (e.g., Scan, FileOpen, FileSave, 
Print, Edit, etc.). Such records are hidden from the user of 
the system (using, e.g., various data encryption and obscur- 
ing techniques), but authorized law enforcement officials are 
provided tools by which these records can be recovered. The 
forensic data thereby obtained may prove useful in pros- 
ecuting counterfeiters. (Knowledge that a computer may be 
covertly storing evidence of attempted counterfeiting 
actions may prove as, or more, valuable in deterring coun- 
terfeiting than the covert records themselves.) 

[0197] The same techniques can be employed to deter 
unauthorized processing of audio, image, video, or content 
by media pirates. In one embodiment, a computer's operat- 



ing system (including peripheral device drivers) monitors 
various data within the system (e.g., data sent to write able 
storage media, or sent via a serial port or network connec- 
tion, etc.) for data bearing a do-not-copy watermark. The 
presence of such data being sent, e.g., to a writeable disk or 
to a remote computer, indicates that the do-not-copy instruc- 
tion has been circumvented. In such case, the operating 
system writes one or more covert records memorializing the 
activity, for possible use in criminal prosecution if the 
computer is lawfully seized. 

[0198] The example just-provided is but one of many 
monitoring and response techniques that may be employed 
to deter circumvention of copy-protection or other access 
control systems. Generally speaking, if content data is found 
where it shouldn't be, or is found used as it shouldn't be 
used, a corresponding record should be made. (Other inter- 
vention actions can be triggered as well; covert tracing is 
desirably just one of several parallel responses to suspected 
hacking.) 

[0199] Me ta -data Accessed Using Watermarks 

[0200] Me ta -data, in formats known as XML, SGML, and 
HTML, is widely used to communicate information about 
digital objects (e.g., author, keywords, price, rights, caption, 
etc.). More generally, met a -data can be thought of as any 
data construct which associates the name of a property (e.g., 
"author), with the value of the property (e.g., "Mark 
Twain"). Such data commonly appears in a tag format, such 
as the following: 

[0201] <META NAME«"author" CONTENT- 
"Mark T\vain"> 

[0202] Meta-data is commonly exchanged between server 
and client computers in conjunction with the digital objects 
to which they relate (e.g., the text of a Mark Twain book). 

[0203] As detailed herein, an important application of 
watermarking is likewise to convey information about 
media — in this case embedded within the media content 
itself (e.g., providing unique identification, establishing 
some basic behaviors such as do not copy, and providing 
links to extended functionality). 

[0204] For meta-data to be useful, it must be linked to 
associated content, whether in the context of a browser, 
application program, operating system, asset management 
system, search engine, etc. However, as detailed below, the 
content and the associated meta-tags needn't always be 
conveyed together. 

[0205] Consider an application program or other client 
process that receives a watermarked media object. The 
watermark includes an MGA for that object (which, as noted 
above, may not specify an ultimate IP address). Stored at the 
MGA site is meta-data corresponding to the object. By 
linking to the MGAsite identified by the object's watermark, 
the client computer can obtain the meta-data corresponding 
to the object. This data can be stored at the client computer 
and used just as any other meta-data, e.g., to define the local 
functions- that should be available for use with that object 
(e.g., buy, search, etc.) 

[0206] A particular example is an on-line catalog of stock 
photography. Each photograph is watermarked with MGA 
data. To identify the photographer, copyright date, price, 
telephone number, subject, etc., an application program can 
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link lo ihe MGA site for that photograph, and obtain the 
corresponding metadata. This data can then be displayed or 
used as needed. Data objects of disparate formats thus can 
readily be handled within a single, simple application pro- 
gram, since the program needn't concern itself with the 
varying formats for the associated meta-data (assuming the 
name servers provide this data in standardized format). 
Substantial flexibility in programming and object formatting 
is thereby achieved. 

[0207] Returning to the internet search engine example 
described above, MGAs may become recognized as reposi- 
tories rich in mcia-data for media objects. Specialized search 
engines may focus their data collection around such sites, 
and be able lo quickly identify (he MGA sites corresponding 
to various boolean combinations of meta-tag parameters. 

[0208] Asset Management; Containers 

[0209] Much has been written on the topic of asset rights 
management. Sample patent documents include U.S. Pat. 
Nos. 5,892,900, 5,715,403, 5.638,443, 5,634,012, 5,629,980 
and laid-open European application EP 862,318. Much of 
the technical work is memorialized in journal articles, which 
can be identified by searching for relevant company names 
and trademarks such as IBM's Crypto lope system, Portland 
Software's Zip Lock system, the Rights Exchange service by 
Softbank Net Solutions, and the DigiBox system from 
InlerTrust Technologies. 

[0210] An exemplary asset management system makes 
content available (e.g. from a web server, or on a new 
computer's hard disk) in encrypted form. Associated with 
the encrypted content is data identifying the content (e.g. a 
preview) and data specifying various rights associated with 
the content. If a user wants to make fuller use of the content, 
the user provides a charge authorization (e.g. a credit card) 
to the distributor, who then provides a decryption key, 
allowing access to the content. (Such systems are often 
realized using object-based technology. In such systems, the 
content is commonly said to be distributed in a "secure 
container.'*) 

[0211] Desirably, the content should be marked (person- 
alized/serialized) so that any illicit use of the content (after 
decryption) can be tracked. This marking can be performed 
with watermarking, which assures that the mark travels with 
the content wherever — and in whatever form — it may go. 
The watermarking can be effected by the distributor — prior 
lo dissemination of the encrypted object — such as by encod- 
ing a UID that is associated in a database with that particular 
container. When access rights arc granted to that container, 
the database record can be updated to reflect the purchaser, 
the purchase dale, the rights granted, etc. An alternative is to 
include a watermark encoder in the software tool used to 
access (e.g. decrypt) the content. Such an encoder can 
embed watermark data in the content as it is released from 
the secure container, before it is provided to the user. The 
embedded data can include a UID. This UID can be assigned 
by the distributor prior to disseminating the container. Alter- 
natively, the UID can be a data string not known or created 
until access rights have been granted. In addition to the UID, 
the watermark can include other data not known to the 
distributor, e.g. information specific to the time(s) and 
manners) of accessing the content. 

[0212] As noted earlier, access rights systems can be 
realized with watermarks without containers etc. For 



example, in a trusting world, copyrighted works can be 
freely available on the web. If a user wishes to make lawful 
use of the work, the user can decode its watermark to 
determine the work's terms and conditions of use. This may 
entail linking to a web site specified by the embedded 
watermark (directly, or through an intermediate database), 
which specifies the desired information. The user can then 
arrange the necessary payment, and use the item knowing 
that the necessary rights have been secured. 

[02 13] Remote Reconfiguration of Watermark Detectors 

[0214] In some cases, it is desirable lo reconfigure water- 
mark detectors remotely. Such functionality is desirable, for 
example, if a watermark system is hacked or otherwise 
compromised. 

[0215] In accordance with this aspect of the present inven- 
tion, some aspect of a watermark detector's operation is 
changed in response lo a command. The change can take 
various forms. In watermark systems employing pseudo- 
random key data (e.g., spread spectrum spreading signals), 
the pseudo-random signal used for detection can be 
changed. In systems using DFI' processing, the mapping 
between message bits and D1T coefficients can be changed. 
In still other systems, the decoding can proceed as before, 
but the significance of one or more bits can be changed (e.g., 
bits that were normally interpreted as defining Field A can be 
interpreted as defining Field B, and vice versa). In yet other 
systems, the decoding can proceed as before, but the 
response of a device to a given watermark signal can be 
changed. In still other systems, a set of software instructions 
can be re -written or re-ordered to effect a change in detector 
operation. 

[0216] The command can be conveyed in various ways. In 
one embodiment, it can be a trigger bit in the watermark 
payload. Normally the bit has a value of "0." If the bit has 
a value of "1," the detector system responds by changing its 
operation. A trigger pattern can also be established, so that 
detection of a certain combination of bits in the watermark 
payload serves to trigger the change. Reserved states of 
certain data fields arc examples of patterns that might be 
employed, 

[0217] The command can also be conveyed through 
another channel different than the watermark channel (e.g., 
an SCA channel of an FM broadcast, or the sub-titling data 
channel of video broadcasts, or header data within an MPEG 
data stream, etc., etc.). 

[0218] The change can proceed in accordance with a 
pre-programmed rule (e.g., codes progressing successively 
through a numerically or algorithmic^ lly-delermined pro- 
gression), or the change can proceed in accordance with data 
specified elsewhere in the payload of the watermark bearing 
the trigger bit (e.g., instead of being interpreted in normal 
fashion, the non-trigger bits of the detected watermark can 
define a new pseudo-random key data. Or the change can 
proceed in accordance with data conveyed in successively- 
presented watermark pay loads, as might be done in video 
encoding where each frame of video can convey further 
watermark information. (This latter arrangement is one 
offering a high-bandwidth re -programming channel through 
which, e.g., extensive firmware instructions might be trans- 
ferred to the detector to replace instructions earlier stored.) 

[0219] By such arrangements, greatly increased detector 
versatility and functionality can Ik achieved. 
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[0220] Conclusion 

[0221] Many diverse embodiments are reviewed above — 
each with a unique set of features. (Still others are disclosed 
in the assignee's patents incorporated by reference.) To 
provide a comprehensive disclosure without unduly length- 
ening the specification, applicants incorporate by reference 
the patents and patent applications referenced above. 

[0222] This specification should be construed as explicitly 
teaching that features illustrated in one such embodiment 
can generally be used in other embodiments as well. Thus, 
for example, a date field was not particularly discussed in 
connection with payload data for video watermarking. Nor 
were "play once" watermarks so-considered. The inclusion 
of a calibration signal with (or as part of the watermark is 
shown in embodiments of the issued patents, but is not 
belabored in the above-described embodiments. Likewise 
with "simple universal codes." The pre-stored commerce 
profile described in one of the foregoing embodiments is 
equally applicable to other embodiments as well. Likewise, 
the presentation of advertising was discussed in connection 
with one embodiment but not others, although it, too, is 
generally applicable. All of these concepts are familiar at 
Digimarc and are regarded as generally applicable through- 
out the work expressed in Digimarc's patent disclosures. 
Practicality prevents an exhaustive recitation of each indi- 
vidual permutation and combination. 

[0223] Having described and illustrated the principles of 
our invention with reference to illustrative embodiments, it 
will be apparent that the detailed arrangements can be 
modified in arrangement and detail without departing from 
such principles. 

[0224] For example, while reference has been made to 
various uses of wireless, it should be understood that such 
reference does not just cover FM broadcast, and wireless 
internet networking and the like, but also includes other 
wireless mechanisms. Examples include cell phones and 
direct satellite broadcast. 

[0225] Likewise, while certain embodiments were illus- 
trated with a watermark payload of 100+ bits, in other 
systems much smaller (or sometimes larger) payloads are 
desirable — sometimes as small as 1-8 bits. 

[0226] While the foregoing examples have each been 
illustrated with reference to a particular media type (e.g., 
video, audio, etc.), it will be recognized that the principles 
of each embodiment find application with the other media 
types as well. 

[0227] Certain of the appliances contemplated above 
require user interfaces more sophisticated than are presently 
typical on such devices. The simplicity of the underlying 
audio appliance can be preserved, in many instances, by 
using a palmtop computer — coupled by infrared or other- 
wise — as a temporary user interface to the appliance. Some 
of the processing capability can likewise be off-loaded to an 
ancillary palmtop. (Palmtop is here meant to refer generally 
to any pocket-size programmable computing device.) 

[0228] Unless otherwise stated, it should be understood 
that the digital music, video, and imagery contemplated 
herein is not of any particular form or format. Audio, for 
example, can be of various forms, both streaming and 
non-streaming, and of various formats (e.g. MP3, MP4, MS 



Audio, Windows Media Technologies, RealAudio, *.WAV, 
MIDI, Csound, Dolby's Advanced Audio Codec (AAC), etc. 

[0229] Having described and illustrated the principles of 
the invention with reference to illustrative embodiments, it 
should be recognized that the invention is not so limited. 

[0230] For example, while digital watermarking typically 
does not leave any human-apparent evidence of alteration or 
data representation, certain of the foregoing applications do 
not require this. The markings used may be visible, and even 
conspicuous, without impairing essential functionality. 
Thus, bar codes, data glyphs, OCR markings, and other 
machine-readable indicia may be substituted, depending on 
the particular application requirements. 

[0231] While the detailed embodiments were generally 
described with reference to desktop computers, it is recog- 
nized that such devices will increasingly be supplanted by 
other digital appliances, including general purpose personal 
digital assistants, multifunction cell phones, and specialized 
devices. Moreover, the power and utility of the above- 
detailed embodiments and devices can be further enhanced 
by employing various wireless communications technolo- 
gies, including the evolving Bluetooth standard. 

[0232] The implementation of the watermark encoding 
and decoding systems is straightforward to artisans in the 
field, and thus not belabored here. Conventionally, such 
technology is implemented by suitable software, stored in 
long term memory (e.g., disk, ROM, etc.), and transferred to 
temporary memory (e.g., RAM) for execution on an asso- 
ciated CPU. In other implementations, the functionality can 
be achieved by dedicated hardware, or by a combination of 
hardware and software. Reprogrammable logic, including 
FPGAs, can advantageously be employed in certain imple- 
mentations. 

[0233] It should be recognized that the particular combi- 
nations of elements and features in the above-detailed 
embodiments are exemplary only; the interchanging and 
substitution of these teachings with other teachings in this 
and the incorporated-by-reference patents/applications are 
also contemplated. 

[0234] In view of the wide variety of embodiments to 
which the principles and features discussed above can be 
applied, it should be apparent that the detailed embodiments 
are illustrative only and should not be taken as limiting the 
scope of the invention. Rather, we claim as our invention all 
such modifications as may come within the scope and spirit 
of the following claims and equivalents thereof. 



We claim: 

1. A method for controlling operation of a device using a 
watermark comprising: 

decoding a watermark embedded in a signal of a media 
type, where the watermark includes a command signal 
usedno trigger a change in operation of the watermark 
detector; 

based on the command signal, changing operation of the 
watermark detector, including changing how the water- 
mark detector decodes or interprets a watermark in a 
signal of the media type; 
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after changing operation of the watermark detector, 
decoding a usage control restriction from a watermark 
embedded in the signal. 

2. The method of claim 1 wherein the command signal is 
represented as one or more bits in a payload of the water- 
mark. 

3. The method of claim 1 wherein changing operation of 
the watermark detector comprises re-programming the 
watermark detector. 

4. The method of claim 3 wherein changing operation of 
the watermark detector includes transferring instructions 
from a remote location to the detector to replace instructions 
stored earlier in the detector. 

5. The method of claim 1 wherein changing operation of 
the watermark detector includes changing the watermark 
detector in accordance with a preprogrammed rule in the 
watermark detector. 

6. The method of claim 5 wherein the preprogrammed 
rule defines a change in a watermark key. 

7. The method of claim 1 wherein changing operation of 
the watermark detector includes changing a watermark key 
used to decode a watermark. 

8. The method of claim 7 wherein the watermark key 
comprises pseudo- random key data. 

9. The method of claim 1 wherein changing the operation 
of the watermark detector includes changing how the detec- 
tor interprets one or more bits of a watermark payload. 

10. The method of claim 1 wherein changing the opera- 
tion of the watermark detector includes changing a response 
of a device to a watermark signal. 

11. The method of claim 1 wherein the command signal 
comprises a combination of bits of a watermark payload. 

12. The method of claim 1 including: 

changing the operation of the watermark detector with 
data provided in a watermark payload. 

13. The method of claim 12 wherein the command signal 
comprises a one or more bit trigger in the watermark 
payload. 

14. The method of claim 12 wherein the data comprises a 
watermark key in the watermark payload. 

15. The method of claim 14 wherein the watermark key 
comprises pseudo-random key data. 

16. The method of claim 12 wherein successively-pre- 
sented watermark payloads in the signal provide the data 
used to change the operation of the watermark detector. 

17. A method for controlling operation of a device using 
a watermark detector comprising: 

receiving a media object and a command associated with 
the media object signaling that the watermark detector 
requires an upgrade to decode a watermark from the 
media object; 

in response to the command, updating the watermark 
detector to create an updated watermark detector; and 

decoding the watermark from the media object with the 
updated watermark detector including decoding a 
usage control restriction from the watermark. 

18. The method of claim 17 wherein the command is 
embedded into the media object in a watermark. 



19. The method of claim 17 wherein the command is 
conveyed in channel different than a watermark channel in 
the media object, and the channel conveying the command 
is transmitted along with the media object. 

20. The method of claim 19 wherein the command is 
conveyed in a header of a file including the media object. 

21. A re -programmable watermark detector comprising: 

a watermark decoder for detecting a command to upgrade 
the detector; and 

detector instructions that are replaceable in response to 
detecting the command to upgrade the detector, the 
detector instructions being operable to decode a usage 
control restriction from a watermark and control play- 
back or recording of a media object carrying the 
watermark. 

22. The detector of claim 21 wherein new instructions are 
conveyed to the watermark detector along with a media 
object. 

23. The detector of claim 22 wherein the new instructions 
are conveyed in a watermark payload to the watermark 
decoder for decoding. 

24. The detector of claim 22 wherein the detector decodes 
watermarks embedded in media objects of a media type, and 
the new instructions are conveyed along with at least one 
media object of the media type in a channel different than a 
watermark embedded in the at least one media object. 

25. The detector of claim 24 wherein the new instructions 
are provided in a media object file that includes a media 
object of the media type. 

26. The detector of claim 25 wherein the media object 
comprises an image signal. 

27. The detector of claim 25 wherein the media object 
comprises an audio signal. 

28. A method of an encoding an upgrade trigger in a 
watermark comprising: 

receiving a media object of a given media type; 

encoding a watermark into the media object, where the 
watermark includes a command signal used to trigger a 
change in operation of a watermark detector, the 
change being operable to change how the watermark 
detector decodes or interprets a watermark in a signal 
of the media type; and 

encoding a usage control restriction into the watermark, 
wherein the change in the operation of the detector 
enables the watermark detector to read the usage con- 
trol restriction. 

29. The method of claim 28 including encoding the 
command in a payload of the watermark. 

30. The method of claim 28 including encoding one or 
more instructions in the payload of the watermark. 

31. The method of claim 28 including: 

transmitting the encoded media object to one or more 
watermark detectors from a location remote to the one 
or more watermark detectors. 

***** 
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